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CHAPTER I 



BACKGROUND AND PURPOSE 

A generally accepted axiom among educational research and devel- 
opment personnel is that the successful planning and completion of a 
specific project depends upon a careful statement of the objectives 
and a detailed procedural outline. To facilitate the process of 
planning and executing educational research and development projects, 
many textbooks and related references on the techniques of research 
methodology have been prepared. While generally helpful in conducting 
the total project, such references are of limited assistance to the 
investigator in conducting the actual day-to-day operations of the 
research and/or development project. 

Detailed directions for the preparation of project proposals as 
established by the Cooperative Research Program and the National 
Defense Education Act of the United States Office of Education and 
similar funding agencies have been of some assistance to research 
investigators in outlining their proposed research activities. Once 
such research projects are underway, however, the investigator may or 
may not conduct the investigation efficiently. 

The development of a procedural technique which would permit 
project directors to determine in advance the significant events and 
activities relevant to the procedural phase of the investigation. 
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and to identify those opera t i ons whose accompl i shment is critical to 
the successful completion of the project, would be very useful. 
Besides initial determination of significant events, continuous re- 
evaluation of project progress through such a technique might 
prevent delays in successful compl et i on of the project. 

At the time of this study, general techniques for educational 
project planning and execution as noted above are of limited availa- 
bility. Other areas of scientific research and development, however, 
have developed techniques designed to economically and efficiently 
accomplish and procedural tasks of the project. The most widely 
known of those techniques is the Program Evaluation and Review 
Technique (PERT) process developed in conjunction with the Polaris 
missile weapons system program. 

Project Purpose 

The success of PERT in military and industrial management and 
control of research and development projects suggests the possibility 
that it might have similar value for educational research and 
development activities. 

\ Consequently, on March 1, 1964, under Office cf Education 
Grant #019* the author undertook a study called the PERT Project, 
with the following set of objectives: 

A. To develop a set of working guidelines for applying 
PERT to a variety of educational research and 
development projects. 

B. To construct a series of model PERT networks for 
common and representative types of educational 
research and development projects. 
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C. To apply the PERT technique to specific on-going 
projects in order to study possible improvement in 
project progress resulting from such application. 

D. To derive a series of recommendations regarding the 
applicability of PERT to educational projects and to 
disseminate them to actual and potential project direc- 
tors through publications, seminars, and workshops. 

The purpose of this report is to describe the procedures that 
were followed in accomplishing the d i ssem i na t i on objective of the 
above study. The importance of disseminating the results of research 
and development efforts has increased during the last several years 
as a means of reducing the '‘theory into practice" gap that is often 
said to exist in the field of education. The staff of the PERT 
project felt that the efforts undertaken to disseminate the knowledge 
gained from the study should be documented and reported to the educa- 
tional community as a case history in knowledge dissemination and 
util i zat ion. 

The total report has been divided into two separate volumes, 
with the first volume dealing with the initial dissemination lectures. 
The second volume. The PERT Lectures: A Case Study in Knowledge 

Dissemination and Ut i 1 i za t i on, Vol . II — Participant Follow-up Study , 
presents the results of a follow-up study conducted approximately six 
months after the initial lectures to determine the extent to which the 
participants utilised the information presented in the lectures. 

Each volume can be read as a separate and complete document. 
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Organization of Report 

The purpose of this first chapter was to present the background 
leading to the dissemination lectures described in this report. The 
procedures used by the author and the project staff to accomplish the 
purpose of the dissemination lectures are reported in the balance of 
this report. 

Chapter II describes the methodological procedures involved in 
planning and performing the lectures. A description of the gathering 
together of the actual content of the lectures as well as a succinct 
description of the content ?or each of the sessions in offered in 
Chapter MI. Chapter IV presents the results of the various evalua- 
tion instruments^ and Chapter V contains a discussion of the results 
along with conclusions and recommendations. Two appendices are 
included at the end of the report to provide supplementary information 
not incorporated in the text. The reader is referred to such infor- 
mation at appropriate places in the text. 
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CHAPTER I I 



PREPARATIONS FOR THE LECTURES 

The purpose of this chapter is to describe the various procedures 
which were involved in planning and conducting the dissemination 
lectures. Specific considerations will be given to publicity and 
selection of sites and participants for the lectures. 

S i te Select ion 

Since the purpose of the dissemination lectures was to inform 
both present and potential educational researchers about the techni- 
ques and direct applications of PERT to on-going research and/or 
development projects, it was of prime importance that the sites for 
the lectures be so situated that the largest possible number of 
persons who would in all probability benefit from implementing PERT 
techniques could attend the lectures. The project proposal conse- 
quently stated that the dissemination lectures be held at universi- 
ties geographically located for their accessab i 1 i ty, numbers of actual 
and potential research personnel in the area, and density of research 
and/or development activity. A letter went out in June of 1964 to 
40 major colleges and universities around the country which might 
serve as possible hosts for one of the twelve two-day dissemination 
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lectures. All of the contacted institutions indicated a willingness 
to serve as hosts. 

The project staff developed a worksheet based upon five criteria 
identified in the project proposal to select the twelve most suitable 
sites. A copy of this worksheet has been included in Appendix A. 

The first criterion was that of resea rch act i v i ty . The project 
staff felt that areas where research and/or development projects 
were either being conducted or planned would naturally contain a 
proportionally higher number of persons who would benefit from 
implementing PERT than would areas which supported little research. 
Therefore, universities situated in an area where large numbers of 
research and/or development projects were being conducted would have 
been assigned a high ranking by the staff. Unfortunately, the PERT 
staff had neither the time nor resources to devise an accurate index 
for identifying the number of research and/or development projects 
in any given area. Consequent 1 y an eas i 1 y appl i ed index had to be 
employed. For this purpose, the 1963 and 1964 programs of the annual 
meeting of the American Educational Research Association were reviewed. 
A tally was made of the number of persons appearing on the program 
from each of the forty universities being considered and the univer- 
sities were ranked accordingly. 

The second criterion was that of travel accessab i 1 i ty , The 
original intent was that each lecture site would function was a 
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regional center serving persons from neighboring states as well as 
persons from within the state. Universities located in a city 
which was served by a major airline were therefore given a high 
ranki ng. 

The third criterion was researcher density within the immediate 
area of the proposed lecture site. Realizing that a lecture conducted 
within 50 miles would draw a bigger audience than one conducted 150 
miles distant, the staff gave a higher ranking to universities 
where there was a high density of researchers in the area although 
they were not necessarily affiliated with the university. 

A fourth criterion for selection, closely related to the third 
cr i ter i on, was geogra ph i ca 1 1 oca t i on . To assure maximum accessabi 1 i ty 
to interested persons from all geographical regions of the country, 
it was necessary to avoid conducting the lectures at two or more 
universities which would draw participants from the same regional 
areas. The staff had several difficult choices to make between 
universities with nearly identical high rankings on the other 
criteria but which would have served the same regional area. 

The fifth and final criterion was the amount of i nterest and 
initial experience with systems technique shown by persons in the 
area to be served by the proposed lecture site. Letters to the 
PERT staff by interested persons, personal contacts, direct solici- 
tations by the staff, and announcements of where systems technique 
presentat ions have been held in the past aided in assigning a ranking 
to the various institutions on this criterion. It was soon 
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discovered, however, that a large degree of ambiguity and error was 
inherent in this criterion. Consequently, the fifth criterion had 
very little influence on the final selection. 

Several highly rated institutions were eliminated as possible 
sites for the dissemination lectures when it was discovered that they 
could not host a two-day conference during the time the PERT staff 
had available. Since it was necessary both to assimilate and develop 
the content of the lectures and to develop a monograph on the appli- 
cations of PERT, the staff had scheduled March and April of 1 965 as 
the period to conduct the twelve dissemination lectures in order to 
realize the August 31* 1965, completion date specified in the 
contract . 

A letter was sent to each institution announcing its selection 
as one of the dissemination lecture sites. The list of final 
lecture sites by date, location, and suggested area of coverage is 
presented in Table 1. 

Lecture S i te , Coord i na t i on 

Once the final selection of the lecture sites had been made 
and an acceptance received from each of the twelve selected univer- 
sities a memorandum was sent to the Dean of the College of Education 
at each host institution requesting that a person be appointed who 
would function as a liaison between the PERT Project staff and the 
host university. 

The liaison personnel or area coordinators had the responsibility 
of making publicity arrangements within the regional area, planning 
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Table 1 - Dates, Locations, and Areas of Coverage for Dissemination Lectures 
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Table 1 - Continued 
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and executing a participant registration setup, and making all 
necessary building and equipment reservations. The liaison personnel 
also supplied the project staff with a list of poss i bl e accomoda t i ons 
for their overnight stay while conducting the two-day lecture and in 
many cases made the actual reservations for the project staff. In 
addition, the materials which were used in the lectures were sent in 
advance to the liaison personnel who in turn mailed unused materials 
back to the project center in Columbus following the completion of 
the lectures at their institution. 

A memorandum was sent in advance to each of these liaison person- 
nel to assist them in making proper arrangements for the lecturers. 
This memorandum outlined what the project staff felt would be needed 
in the way of lecture room facilities, audio-visual equipment, publi- 
city, and registration arrangements for. participants. It was also 
pointed out in the memorandum that the PERT project only had funds 
to cover travel expenses for the lecture personnel and for materials 
to be distributed at the lecture so all room and equipment rentals 
or incidental expenses would have to be the responsibility of the 
local institution. 

Participant Selection 

Although the dissemination lectures were primarily intended 
for persons actively engaged in educational research and/or develop- 
ment, no attempt was made to select participants by any formal means. 
As stated in the proposal, a primary objective of the lectures was to 
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inform not only persons actively engaged in educational research but 
also other persons who could potentially benefit from utilizing 
PERT techniques. Thus the lectures were open to any persons interested 
in the general substance of the lectures. 

Since one of the stated objectives of the lectures was to inform 
and since the costs of the project would remain somewhat constant 
regardless of the number of participants (only the cost of handout 
materials would fluctuate), no limits were placed upon audience size 
at any of the lectures. For planning purposes, it was anticipated 
that the maximum number of participants at any one session would be 
100 persons. 

Publ icity 

The size and composition of the audiences at the various lecture 
sites was dependent to a great degree upon the effectiveness of the 
publicity phase of the project. Several different procedures and 
techniques were used in order to publicize the lectures. The basic 
procedures used were the development of a publicity brochure and 
news releases. 

A publicity brochure or flyer was developed in conjunction with 
the Publications Office of the College of Education, The Ohio State 
University. Information regarding dates and places along with the 
general purpose of the lectures was included in the flyer. These 
flyers were printed on a gray paper with red lettering on one side 
for easy folding and addressing purposes. 
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Since the American Educational Research Association is the 
largest organized body of educational researchers, it was considered 
a prime target for publicity on the dissemination lectures* The 
flyer was distributed to the attendees at the 1 965 AEixA convention 
in Chicago by placing approximately 500 of them on the registration 
desk. Fifty copies of the flyer were also sent to each liaison 
person at each dissemination lecture site for regional distribution 
and a supply was also forwarded to the U. S. Office of Education for 
their internal and external distribution. 

The project staff enlisted the assistance of The Ohio State 
University News Bureau to help prepare a news release on the dissemina- 
tion lectures. This news release was sent to major newspapers in 
the areas to be served by each of the regional lecture site centers. 
The OSU News Bureau also assisted in the formulation of a sample 
release which was sent to each area coordinator or liaison personnel 
to provide them with specific content for any publicity they would 
be sending out within their regional area. In addition, a chart was 
sent to each liaison man outlining the geographical extent of the 
regional area that he was responsible for covering in his publicity 
efforts . 

Publicity announcements were also placed In various professional 
newsletters and journals, such as the AERA Newsletter , whose reader- 
ship would be primarily educational research and/or development 
people. A list of the publications which were solicited to carry 
these announcements as well as the information sheet which was sent 
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to help them in writing the announcement as it would appear in their 
publication is included in Appendix A. 

It was discovered during the course of the dissemination lectures 
that publicity efforts by the regional coordinators were predominantly 
in the form of personal contacts such as letters or phone calls. It 
appeared, moreover, that when an area coordinator sent out a formal 
announcement, it was generally sent to a group with which the coordina- 
tor was affiliated. In fact, a clear relationship seemed to exist 
between the professional specialization of the area coordinators and 
the aud i ence a t their respective dissemination lecture. At each 
center, the size and composition of the audience appeared to be 
dependent upon the particular interest and background of the coordina- 
tor assigned by the institution. For example, one center was 
coordinated by a person who primary affiliation was with educational 
administration. Consequently, the lecture announcement was distri- 
buted primarily to persons involved in school administration. Thus, 
if a coordinator was engaged primarily in research the audience 
usually had a large proportion of researchers but if the coordinator 
was an administrative specialist the audience tended to be composed 
of administrators. The observation noted above suggests that in 
future dissemination activities, such as the PERT lectures, careful 
consideration be given to the background and position of area 
coordinators in order to have reasonable assurance that the appropriate 
audience is being reached* 
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CHAPTER 111 



DISSEMINATION LECTURE PREPARATION AND CONTENT 

This chapter describes the act i v i t i es involved in the prepara- 
tion of the lectures along with an overview of their content. For 
convenience in presenta t i on, separate sections have been developed 
for the preparation and the content. 

Prepa rat ion 

The development, preparation, and presentation of the dissemina- 
tion lectures was undertaken primarily as a joint responsibility of 
the project director and the assistant project director. The 
development and preparation of the lessons utilized several sources 
of materials in addition to the project staff. One major source was 
the Instructors Manual on PERT prepared by the PERT Orientation and 
Training Center i n Wash i ngton, D. C. This manual provided illustra- 
tive lesson plans for various topics along with recommended student 
activities. A set of two-by-two slides was also available for use 
with this manual. These slides were quite helpful in preparing 
subsequent overhead transparencies. The five volume Air Force series 
on PERT was also heavily utilized in developing lesson content. The 
project staff also had available to it a series of lesson plans from 
North American Aviation Company which were developed by its Training 
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Divisions and made available to the project by a consultant. During 
August of 196*+ Mr. Don Frost, an instructor at the PERT Orientation 
and Training Center, consulted with the project staff on lesson 
preparation and gave invaluable assistance in lesson development. 

in preparing the lectures, it was necessary to operate within a 
time constraint of a two-day period because of an earlier decision to 
provide only an orientation within such a period of time. Considera- 
tions with regard to content and activities of the dissemination 
lecture, therefore, always had to be evaluated in terms of the contri- 
bution of the topic to the overall purposes plus its immediacy and 
potential usefulness to the lecture participants. The advice provided 
by the consultants with regard to a general time frame for both the 
individual lesson topics and general lesson outline proved very 
helpful to the project staff since no extended effort was made to try 
out the lessons prior to their actual presentation. In retrospect, 
it was found that there was a time overrun on the first series of 
presentat ions but later adjustments deleting some material resulted 
in an appropriate time frame being maintained. 

The steps taken in developing an actual lesson consisted of 
developing a lesson plan worksheet which provided space for the 
session number, the instructor's name, the visual aids to be used, 
the time length of the presentation along with the topic. Preliminary 
outlines were made for each lesson and then reviewed by the project 
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staff and consultants. Both the project director and assistant 
project director attended a week-long course at PERT Orientation 
and Training Center, Washington, D. C. 

A decision was made early in the lesson preparation stage to 
use overhead transparencies rather than two-by-two slides. The 
primary reason for choosing this media form was the fact that 
available slide sets were inappropriate to the field of education* 

It was, therefore, easier to develop new or modify existing trans- 
parencies so as to make necessary illustrations more relevant to 
educational personnel* In most cases, drafts of the transparencies 
were prepared by the project staff and final completed drawings made 
by the Teaching Aids Laboratory of The Ohio State University. In 
some cases, the transparencies were not visible in large auditoriums, 
so efforts were made during the course of the lectures to change to 
a more appropr ia te size for viewing. As a general rule, participants 
did not receive copies of the visuals although many participants 
requested them. Some effort was made to meet this demand during and 
after the lecture series was completed but the task of production 
and distribution became more than the limited project staff could 
handle. 

The project staff had utilized during some early local orienta- 
tionand training sessions for educational personnel several films 
developed by the military and industry which provided the basic 
principles of concepts of network techniques. Of the several films 
available and exhibited from time to time, an orientation film of 
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about fifteen minutes in duration produced under the direction of 
North American Aviation Company^ was found to be highly acceptable 
by untrained personnel. The decision was made therefore to incor- 
porate this film into the training program. A second film running 
about 30 minutes and covering the basic concepts of PERT/COST was 
also accepted by untrained personnel and it was also incorporated into 
the lecture series. 

The project staff decided that it would be quite advantageous to 
the dissemination process if participants could take home with them 
some item other than the notes that each participant might take. 

Two such items were obtained and/or developed for this purpose. The 
first item was a publication produced by Douglas Aircraft Corporation 
titled Why Play It By Ear - Use PERT . This pamphlet had been called 
to the attention of the project director and a request was made of 
Mr. Hal McDonald of Douglas Aircraft Corporation for a supply 
sufficient to meet the anticipated audience of the dissemination lec- 
tures. This request was granted and arrangements were made to include 
this pamphlet for distribution at the lecture sites. 

The second item was a n date calculator 11 which had been developed 
by personnel at North American Aviation. This device consisted of an 
inner and outer wheel which allowed a person to coordinate time on the 

^The North American film was obtained from General Film Labora- 
tories, 1546 Argyle, Hollywood, California for approximately $83.00. 
The PERT/COST was obtained from Duart Film Laboratories, 245 West 
55th Street, New York at a cost of $93.00. 
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inner with calendar dates on the outer wheel by appropriate turning 
of the two wheels. The original date calculator was constructed of 
aluminum but the ones prepared for the dissemination lectures were 
made of a stiff cardboard. Both of these items were distributed at 
appropriate points during the dissemination lectures presentation. 

The project staff*s prior exper i ence wi th PERT orientation plus 
attendance at various orientation sessions pointed up the necessity 
to have some kind of a simple exercise that participants could work 
on in order to gain greater understanding of network concepts. An 
exercise was developed which derived mainly from survey research 
methods and involved a projected study of faculty attitudes towards 
parking on a university campus. initial drafts of the exercise were 
prepared and then reviewed by consultants. The total exercise was 
originally divided into five separate subexercises with stress put on 
particular topics such as activity identification, network construc- 
tion, Expected Elapsed Time calculation. Earliest Expected and Latest 
Allowable date calculations, scheduling, and replanning. No attempt 
was made to build into the practical exercise any work relating to 
cost or budget determination. The practical exercises were duplicated 
in quantity and shipped in advance to each of the dissemination lec- 
ture sites for distribution at the proper time. 

All of the lesson plans were prepared in triplicate copy. A 
notebook was made for each of the instructors with the third copy 
being maintained in the file for safety and record purposes. 
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Content 

This section provides only a brief but succinct description of 
the content included in each of the major sessions. It is not 
feasible to present the lectures in deta i 1 nor to provide duplicate 
copies of the lesson plans as appendix materials for two reasons. 
First, lesson content and sequence were under continuous modification 
during the course of the lectures. Second, the inclusion of the 
original plus modified material would make the total report lengthy 
and bulky without serving any substantial purpose. 

The overall two-day lecture presentation was divided into four 
general sessions. The first session dealt with the nature of manage- 
ment and basic PERT terminology and concepts. The second session 
was devoted to the practical exercise and sample problem. The 
third session presented some applications of network techniques to 
educational research and development of projects, the fourth session 
was devoted to PERT/COST plus a review and summary. Additional 
information regarding subject matter and activities for each of the 
sessions is described below. A copy of the general outline distri- 
buted to participants is included as Table 2. 

First Session (A. M.) - The initial activities in the first 
session were devoted to courtesy matters involving the host institu- 
tion as well as expressions of appreciation to the coordinator, 
followed by the introduction of the PERT project staff. A discussion 
of participant materials that had been distributed during a registra- 
tion period along with an outline of the lecture was then made. 
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Table 2 - Dissemination Lecture Outline and Schedule 



TIME 


TOPIC 






FIRST SESSION 


8:45 - 


9:00 


Regl st rat Ion 


9:00 - 


1 1 :45 


Introduction to Management and PERT 

The Management Process and Educational 
Resea rch 

Basic PERT Terminology 
Network Construction 
Time Est imating 
Schedul ing 
Probab i 1 i ty 


12:00 - 


1:30 


Lunch 


1:30 - 


4:30 


SECOND SESSION 






Pract i ca 1 Exerci ses 

Activity Ident i f icat ion 
Network Construction 
Time Estimating 
Schedul i ng 






Project Replanning 


9:00 - 


11:45 


THIRD SESSION 






Applying PERT to Educational Research and 
Development Projects 

PERTing the PERT Project 






Applications to Educational Projects 
Model Networks 
Project Applications 






Suggestions for Implementing PERT 


11:45 - 


1:30 


Lunch 
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Table 2 - Continued 



TIME 


TOPIC 


! : 30 - 4:00 


FOURTH SESSION 




Introduction to PERT/COST 




Review and Summary 




Questions and Answers 




Lecture Evaluation 




Award Certificates 
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The initial topic was a discussion of the relationship of the 
management function processes to educational research, particularly 
developing a rationale for the study of management systems that might 
apply to educational research and development. Such basic concepts 
as the management cycle, planning, the role of objectives, the inter- 
relationship of time-cost-performance were presented. After this 
presentation, the introductory PERT f i lm was shown. Following the 
showing of the film, the basic PERT concepts and principles were 
then presented. Such terms as event, activity, network, critical 
path, and slack were discussed. After this general orientation to 
terminology, a presentation was made on worbreakdown structure 
development, including hierarchical nature of objectives. The proce- 
dures involved in developing networks including their functions and 
use, managerial aspects, network terminology, and rules of network 
construction were then reviewed. After a presentation of the work- 
breakdown structure and network construction, information relating to 
time estimation for activities and events was then presented, including 
some discussion of statistical assumptions. Attention was paid to 
calculating Expected Elapsed Time for an activity. Earliest Expected 
and Latest Allowable time for events and critical path. Following 
this topic, some discussion of the role of scheduling utilizing the 
time estimation information was made. A presentation followed on the 
role of computers in processing network calculations, followed by a 
brief discussion of the probability dimensions of PERT. A review and 
summary completed the presentation for the first session. 
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Second Session (P. M.) -This session was devoted almost entirely 
to the practical exercise referred to above. The purpose of the 
practical exercise was explained and a review of PERT terminology 
was made. The first item was devoted to developing a workbrea kdown 
structure. The second item involved constructing a network. The 
third item was devoted to calculating activity times using multiple 
estimates. The fourth item was devoted to a calculation of the 
Earliest Expected Time (T £ ) and the Latest Allowable Time (T ) . The 
fifth item was devoted to creating a schedule using the date calcula- 
tor described above. The final item was devoted to a necessary 
replanning in view of certain problems which developed during the 
course of the project. The purpose of the latter exercise was to 
reinforce the decision-making in which the project director must 
participate. Each exercise had a problem sheet and a ’'school 11 solu- 
tion sheet except for the last two exercises. Opportunity was pro- 
vided to discuss each of the solutions. 

Third Session (A. M.) - This session was devoted to the discus- 
sion of the application of PERT to educational activities. The nature 
and purpose of the PERT project itself was discussed along with the 
major objectives of the project. This introduction was followed by 
a description of how PERT was applied to the PERT Project including 
the initial development of the network and workbrea kdown structure 
and selection of a computer program for network processing. Examples 
of networks developed and changes made during the course of the 
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project, showing how information was processed and decisions made, 
was also included. Sample computer printouts were distributed as 
part of this session. 

The next topic presented was a discussion of the development of 
model wor kbreakdown structures and networks for various types of 
educational projects including such types as experimental, survey, 
test development, historical, and similar efforts. Illustrations of 
a workbrea kdown structure and network for each of the several types 
were then presented. 

Following the presentation on model networks for various types 
of research and development activities, the application of PERT to 
specific projects at Ohio State was then discussed. It was orginally 
intended that the project directors involved in these applications 
would be part of the dissemination lecture staff and would describe 
the application to their particular projects. This was actually 
followed in one instance where Dr. Daniel Stufflebeam gave a discu- 
sion of the revision of the GED Tests during the presentation at 
Florida State University. The primary reason for modifying the 
original approach was based upon the fact that it would have required 
a staff member to be gone for a week in order to make approximately 
two one-hour presentations. The PERT project staff questioned 
whether or not this would be a wise expenditure of a professional 
person's time. Details were presented regarding the initial proce- 
dures and computer application for each of the presentations showing 
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some of the problems involved. This topic was terminated by pro- 
viding evaluative information regarding the project application 
phrase along with a summary of responses made by the project staff 
d i rectors . 

In addition to the applications described above, the partici- 
pant 1 s a ttent i on wa s called toother types of a ppl i ca t i ons that 
were known to the project staff such as the establishment of junior 
colleges, the installation of computers, and similar applications. 

The session was concluded by outlining some suggestions for imple- 
mentation of PERT in a specific situation. Attention was given to 
general considerations such as the specific steps that should be 
f ol lowed . 

Fourth Session (P, M.) - The fourth session was devoted primarily 
to two major topics. The first was an introduction to the concept 
of PERT/COST. The second was devoted to a review and summary. The 
basic presentation on PERT/COST was made utilizing a set of slides 
available from the PERT Orientation and Training Center. Historical 
development of PERT/COST was reviewed along with the reasons for its 
development. In addition to the set of slides, a film developed by 
the PERT Coordinating Group in Washington on the nature of PERT/COST 
was shown to introduce the concepts. Emphasis was given to relating 
budgets to work or output categories rather than to traditional input 
categories . 

The remaining portion of the fourth session was devoted to a 
review and summary of the content presented during the two-day 
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period. One essential dimension of this summary was comments by 
the project staff on the validity of PERT for educational research 
and the development activity both in the planning and controlling 
stages of a project. 

This session was approximately two hours in duration. It was 
necessary to terminate the fourth session early at the end of any 
two-day orientation in order that the project staff could either 
make transportation connections to the next site or return to home 
for the weekend. 

The general sequence of content presentation was generally 
satisfactory except for two major ccmments made by the participants. 
The first comment centered around a suggestion that the practical 
exercises be correlated with the substantive presentation in order 
that a particular concept have immediate reinforcement. A session 
devoted totally to content followed by exercises turned out to be 
inadequate learning experience. The second comment focused upon the 
short time given PERT/COST in the fourth session since no discussion 
in depth could be presented. A more detailed evaluation by the 
participants of the total training session is contained in the 
next chapter. 
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EVALUATION OF THE LECTURES 

No specific contracutal requirement existed to present evidence 
of either an objective or subjective nature relative to the success 
or failure of the dissemination project. The project staff decide^ 
to Implement assessment procedures which would provide evidence to 
revise the presentat ion. 

Measurements of substantive achievement were not attempted since 
the stated purpose of the lectures was to make researchers in educa- 
tion aware of the possibilities of PERT as a management tool. Had 
the purpose of the lectures been to train persons in the use of PERT 
rather than just to orient them to the technique, then measurements 
of achievement would have been appropriate. Several objective 
and subjective evaluations, representing different frames of reference 
were made and are described in this part of the report. Opinions of 
the participants about each lecture were secured immediately after 
each of the two-day programs was completed. The project director 
recorded a reaction to each lecture location including any changes 
made in the basic lesson outline for that lecture series along with 
a description of the facilities, arrangements and audience. An attempt 
was made to evaluate the d f sstuii nat ion lectures on the basis of 
conformity to the budget allocations which were planned at the onset 
of the project. The cost analysis report can be viewed as a case 
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study evaluation of the cost/effect i veness of a knowledge dissemina- 
tion project such as that represented by the dissemination lectures. 
Each of these evaluations is discussed in detail in subsequent sections 

Participant Evaluation 

At the conclusion of each two-day dissemination lecture, partici- 
pants were given a questionnaire which they were to complete immediate- 
ly. The questionnaire consisted of eleven s tructured-response items 
concerned with the substance of the session and one structured-response 
item asking for an evaluation of the instructors in terms of content 
knowledge and effectiveness of presentation. A copy of the question- 
naire is included in Appendix A. 

Generally speaking, the participants gave a very positive 
evaluation to the dissemination lectures. They indicated that the 
content of the lectures was both pertinent and useful to them and that 
the PERT staff did a good job of organizing and presenting the lectures 
A summary of participant evaluatiDns over all sessions is presented 
in Table 3. The total number of responses to individual items varies 
because some participants did not respond to all of the items. 

Of the 247 participants who responded to the item concerning 
pertinence of the lectures to participants* work duties , 22 per 

cent indicated they had relevance. Almost 44 per cent of the partici- 
pants indicated a significant degree of pertinence, slightly over 
27 per cent indicated that the content was only somewhat pertinent to 
their work, while the remaining participants indicated that the con- 
tent either had very little or no pertinence to their work. 
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How effective was the use A Great Deal 65 25.0 
of visual aids (slides, movies. Significantly 156 60.0 
handouts) in the lectures? Somewhat 3k 13^1 

Very '.ittle k 1.5 
Not At All 1 .4 
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Although 93 per cent of the participants had indicated the 
lectures were in some degree pertinent to their work, only 55 per 
cent (based on 255 responses) noted a direct relationship between 
their work duties and the lecture material while 4l per cent indicated 
only an indirect relationship. The remaining 4 per cent of the 
participants indicated either that their work was not complex enough 
to warrant the use of PERT or that they held a teaching position 
where they would not have occasion to utilize PERT. Several of these 
participants were graduate students who attended the lectures out of 
curios i ty . 

Almost 94 per cent of 259 participants indicated that they anti- 
cipated making some use of th? material presented in the lectures. 

About 12 per cent of these participants anticipated using the lecture 
material a great deal with over 40 per cent anticipated making signi- 
ficant use of it. Another 42 per cent indicated that they planned to 
use the lecture material somewhat. Again, there were a number of 
participants (6 per cent) who indicated that they were not involved in 
any of the management functions of educational research and/or develop- 
ment and therefore would have very little or no occasion to use the 
1 ecture material . 

The use of visual aids during the lectures was considered effec- 
tive by all but 2 per cent of the 260 participants who responded to 
this item. A fourth of the participants rated the effectiveness of 
the visual aids as great and 60 per cent rated it as significant. 
Although the participants almost unanimously evaluated the visual aids 
as effective in some degree, they also had numerous suggestions for 
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improvement. About 2 per cent of the participants suggested that 
future transparencies contain larger print so that they could be 
more easily read. Several participants further suggested that future 
transparencies have more color in them and better contrast. Another 
2 per cent suggested that the project staff distribute handouts over 
the material presented by the slides. 

All but six out of 246 participants felt that the practical ex- 
ercises were of some value to them. Almost 55 per cent of the parti- 
cipants indicated that the exercises were of significant value to 
them, while 28 per cent said that the exercises were of great value. 

Of the six participants who indicated that the practical exercises 
were of little value - one person said that they were too difficult 
to be of much value for him, another cited the exercises as being too 
easy. Two participants said they had no intention of using PERT and 
the exercises were irrelevant to them and two other participants offer- 
ed no comments on why the exercises had little value for them. Alto- 
gether, over 5 per cent of the participants suggested that more time 
be allowed to work on the exercises, and another 5 per cent also 
suggested that more explanation be given on how to complete the exer- 
c i ses . 

it would seem from the response to the above item that a number 
of the participants found the practical exercises difficult although 
of some value. However, the practical exercises were considered to 
be of average complexity by 69 per cent of 248 participants, with only 
one person rating them as too difficult. Another 9 per cent consider- 
ed the exercises to be difficult, 20 per cent found them easy, and 
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2 per cent considered them to be too easy. Several participants did 
suggest that the number of computations involved in the exercises 
be cut down. It was suggested by several participants that the exer- 
cises could be given throughout the lectures rather than during just 
one time segment. It was thought that in this manner the exercises 
could give more immediate reinforcement of the PERT principles and 
at the same time would not become boring to the participants. 

The majority of the participants, 58 per cent of 248, also rated 
the level of detail in the lectures as being about average, with an- 
other 31 per cent considering the lectures detailed, and 7 per cent 
ranking them as being too gross. 

The overall rating of the lectures was high, as 34 per cent of 
244 participants gave a rating of excellent to the overall quality 
°f the lectures. Another 59 per cent of the participants rated the 
lecture as good and over 5 per cent rated them as average. The most 
prevalent suggestion (approximately eleven times) for improvement was 
to eliminate some of the redundancies in the various lectures. There 
were almost as many observations (ten times) that the lectures would 
have been better if some of the examples and illustrations had been 
taken from an educational context rather than a military one. Sever- 
al participants also indicated a desire for more handouts over the 
lecture material . 

Reaction was mixed on the question of omitting one session from 
the lectures. Over half of the 167 participants who responded to 
this item favored omitting the fourth session (Introduction to 
PERT/Cost); almost 38 per cent favored omitting the third session 
(Application of PERT Technique to Educational Activities). However, 
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a full third of the total number of participants left this Item blank 
with nearly all of them writing In that none of the sessions could be 
om I tted . 

The participants split even on the question of lengthening the 
lectures so PERT/Cost could be considered In more detail with slight- 
ly less than 52 per cent of 210 participants rejecting the Idea of 
allowing more time for PERT/Cost. Several participants Indicated that 
additional PERT/Cost material would be desirable but not In the Ini- 
tial presentations . They suggested th*t a follow-up lecture be con- 
ducted on PERT/Cost since many of those attending the general PERT 
lectures would have little need or Interest In PERT/Cost. 

The first session was rated by 43 per cent of 245 participants as 
being the most Informative followed closely by the second session 
with 38 per cent of the participants favoring It. The third session, 
which on an earlier Item had been selected for omission from the lec- 
tures by 38 per cent of 167 participants, was rated as the most in- 
formative session of the lectures by over 15 per cent of 245 partici- 
pants. Several participants indicated that since they had come to the 
lectures with the sole purpose of finding out the nature and uses of 
PERT, the first session was the most informative to them. 

The instructors fared quite well on the final question which 
evaluated them in respect to their presentation, knowledge of the 
subject and ability to explain and clarify points. Over 57 per cent 
of 249 participants rated the instructors as average, with the excep- 
tion of one person who rated them as poor on presentation but excel- 
lent in the other categories. 
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Responses to each of the twelve items categorized by lecture 
site are presented in Appendix B. The general pattern noted above 
was also exhibited In the responses from the individual locations. 

Based upon the evidence obtained from the participant evaluation, 
It can be summarily stated that the lectures were well accepted by 
those persons attending them. Several suggestions advanced by parti- 
cipants for improvement of the lectures were eventually incorporated 
as may be noted in the following section on the project director's 
reports . 

Lecture Site Evaluations 

This section of the report Is devoted to a series of short des- 
criptions of reactions recorded by the project director at the actual 
site of the regional dissemination lectures. Each regional lecture 
and lecture site is described separately to show their interrelated- 
ness. With one or twominor ed i tor ial changes , the reactions presented 
below are verbatim copies of the original notes. 

University of California (Berkeley) . (Monday-Tuesday , March 1-2). 
The lectures were conducted at Tolman Hall. Approximately 40 parti- 
cipants attended the morning session on Monday, and 25 participated in 
the afternoon session that same day. Attendance on Tuesday was about 
27 for both sessions. 

The morning session on Monday started about 9:15 with an intro- 
duction by Dean Keller, Dean of the College of Education. The script 
was then covered through to the movie. A five to ten minute break 
was taken at 10:30. Due to the delay in starting, some of the mater- 
ial had to be covered too quickly. Also, some material tended to be 
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repetitious while other material was misplaced in terms of the topic. 
The morning script was revised so as to allow more time for emphasis 
on basic points. Other than the above points, the morning session 
went well. 

The afternoon session also went very well. One or two small 
errors appeared in the problems. The participants felt that the 
practical exercise was quite worthwhile. The afternoon session was 
concluded about 4:15 with a quick summary. The sessions on Tuesday 
went according to the script and few problems were encountered. 

Wednesday, March 3, was spent on revising the materials before 
departing for the University of Southern California. This initial 
lecture emphasized the need to either start the first session prompt- 
ly or move the rest of the program back so as to adjust for a late 
£ tart. 

University of Southern California . (Thursday-Fr i day , March 4-5). 
Registration was very light at USC with only 12 persons in attendance. 
Most of the participants were from agencies such as the United States 
Air Force and System Development Corporation but within such agencies 
many had responsibility for educational activities. 

The Thursday session, March 4, was conducted according to the 
script as revised after the Berkeley lectures. The revised session 
went quite smoothly. On Friday the morning session started late (15 
minutes) and as a consequence some of the morning material carried 
over to the afternoon session. The staff received several requests 
for copies of the visuals as had happened at Berkeley. 

Florida State University . (Monday-Tuesday , March 15-16). About 
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45 persons attended the lectures conducted in the Student Union build- 
ing at Florida State University, The registration setup was the best 
encountered so far but the arrangements for the visuals were not too 
good although satisfactory. The first session ran a bit later than 
usual (until noon) consequently the final portion had to be hurried. 
The afternoon session seemed to go pretty well. 

On Thursday about 30 participants still remained despite the 
fact that several had previous experience with PERT. The presentation 
on Tuesday was varied slightly in order to see if the PERT Project 
application could be included as part of the project application 
phase. The revision did not appear to be satisfactory so the old 
presentation form was retained. The PERT/Cost session was hindered 
by difficulties with the projector. The light from the projector 
was completely insufficient and the room had to be darkened to the 
point where participants could not take notes. Lesson learned: 
check all slides the first day so adjustments can be made if they do 
not project. 

It became apparent after this lecture that Practical Exercises 
No. 5 and No. 6 needed revision. The scheduling of activities show- 
ing slack was added to Practical Exercise No. 5, thus making it more 
consistent with the principles of scheduling. Practical Exercise No. 

6 was revised to ask participants to replan according to selected 
decisions (e.g. , add more key purn.h, eliminate tryout). 

Syracuse University . (Monday-Tuesday , March 22-23) • Approximate- 




ly 60 persons attended the lectures, which were held in the Northway 
Motel on the outskirts of Syracuse. The setup at Syracuse was very 
good and represented a great deal of thought and work by the host 
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institution. Participants were there from New York, Vermont, Mass- 
achusetts and New Jersey; they all appeared to be highly interested 
in PERT. The session on Monday, March 22, followed the same presen- 
tation as was used at USC and went quite smoothly. 

The morning session on Tuesday started late due to interest in 
the Project Gemini launch that morning. Consequently, some of the 
detailed presentation on the project application phase was deleted. 

After lunch the staff presented some suggestions for implementing PERT. 

Pictures were taken during the sessions and were carried on 
television on the evening of March 23. The size of the audience and 
the room at Syracuse gave convincing evidence that some of the visu- 
als should be prepared also as handouts. A call was made to the staff 
in Columbus to put this process in motion. 

Columbia University . (Thursday-Friday , March 25-26). An aud- 
itorium in Macy Hall which seated 150-157 persons was the site for 
the Columbia lecture. The audio-visual set-up was good and the 
registration procedures were also quite satisfactory but attendance 
was small with only 15-20 participants. Some confusion seemed to 
exist regarding publicity. Some persons in the city area were not 
made aware of the conference and heard about it only through Phi Delta 
Kappa and other national sources. A couple of participants were from 
Albany and attended upon the recommendation of persons attending the 
lectures at Syracuse. 

The morning session on Thursday, March 25 contained several changes. 
Several of the visuals were eliminated and the introduction to PERT 
was shortened, using the management cycle as a means of orientation. 
Introductory material on educational research and development was to 
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be shifted over to the morning session on Friday. Even with the 
visuals reduced, the morning session was still long but did seem 
to go better. Computer printout sheets were introduced after the 
practical exercise in order to illustrate both computational pro- 
cedures and types of output reports* The printouts seemed to be well- 
received. 

The morning session on Friday was conducted as previously with 
little change. However, more detail than usual was included in the 
project appl ications phase and as a consequence the material on ed- 
ucational research and development which had been shifted from the 
Thursday session did not get included. Implementation steps were 
also expanded in the afternoon session. Several new slides on PERT/ 
Cost were also included and seemed to help clear up some ambiguities. 

The confusion over responsibility for regional publicity suggests 
that in the future a form letter should be sent to the liaison men 
asking them what publicity arrangements they have made and empha- 
sizing that they are responsible for assuring a good audience. 

Washington University . (Monday-Tuesday , March 29-30). The 
room for the lectures was Steinberg Hall, a fair sized auditorium. 

The registration procedures were well executed but again the atten- 
dance was small with about 20 participants. About one-third of the 
participants were from a nearby Air Force installation. There were 
few from Washington University since the school was then in the se- 
mester break and most professors had left for a few days. 

The sessions on Monday were held as previously. It was decided 
to adjourn at 3:15 P. M. due to prior commitments on the part of a 
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majority of the participants. 

The morning session on Tuesday started slightly late due to 
delay in transportation from the motel where the staff was staying. 
Consequently there was little time to organize before starting. A 
completely new organization of the material was attempted, using che 
management cycle as a way of organizing the material on project 
application. The sequence was a little loose and the group was not too 
enthusiastic, possible because of their own prior experience with 
PERT and/or lack of interest in education activities. It was 
decided to try the same organization at Iowa to see how they would 
rer _,d. 

The afternoon session on Tuesday was late getting started, which 
caused some problems in holding the attention of the group. Also, 
the wind-up was too brief. A good termination point, c t than a 
review of what was already covered, seemed to be needed. 

University of Iowa . (Thursday-Friday, April 1-2). The first 
session of the lectures at the State University of Iowa started on 
time in Butler Hall with approximately 60 persons attending. The 
majority of the participants were school adm i ni s t rators although there 
were also some persons there from Iowa State University. Excellent 
seating arrangements and good facilities for coffee breaks helped the 
sessions. The reduced number of visuals was used again and worked out 
well. The morning session concluded at 11:45 A. M. 

The afternoon work session went well considering the group size, 
although there was some difficulty keeping quiet since they finished 
at different times. The final activity of the afternoon session was 
computer processing of PERT data. 
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Friday's morning session convened on time with the material 
presented in the same format as at Washington University, organized 
around the steps of the management process or cycle. It appeared 
to work well except that networking and scheduling could have had 
more "appl icat i ons M . Also, illustrative examples of activity and 
event cards would have been useful. 

The afternoon session was a bit late in starting, however it 
was finished by 3:00 P, M. The staff noted that here, as with all the 
other lecture sites to date, there was a slight problem of obtaining 
steady attention. The third and fourth sessions were always less 
crowded than the first two. 

University of Minnesota . (Monday-Tuesday , April 5-6) • The 
lectures, held in the Coffman Memorial Union, were conducted at the 
University of Minnesota in the same format as at the University of 
Iowa. The attendance, however, was only six in the morning session 
and two in the afternoon session. The attendance for Tuesday was 
limited to four at both sessions. 

In checking the reasons for the lack of participation it was 
discovered that the area coordinator was not clear on exactly how 
the publicity was to be handled. He appeared to assume that the PERT 
staff would send publicity notices to all schools in the geographical 
area. Consequently, he did not send any notices to schools outside 
of the Minneapolis area. In fact, since all of the participants were 
administrators at the host university, it appeared that notices were 
not sent outside the university. The coordinator admitted that per- 
haps he she ild have investigated the publicity angle but claimed that 
a lack of office staff would have made it impossible to assume any 
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publicity responsibilities anyway. The need for more communication 
about publicity responsibilities again became quite clear to the 
Project staff. 

University of Wisconsin . (Thursday-Fr iday , April 8-9). Regis- 
tration procedures were very well handled at Wisconsin and it 
appeared that much effort had gone into the publicity and related 
activities to insure a good audience. Even with rainy weather 
approximately 60 persons attended, many coming from places outside of 
the state. 

The morning session on Thursday went well, although several 
changes were made in the script. The new bulletin on computer print- 
outs was used for the first time and seemed to help very much. The 
two handouts on event/ac t i v i ty dependencies were not used because 
they were too incomplete and required too much hand drawing to be 
done by the group. The afternoon session went according to the re- 
vised script and was ended, as usual, about 4:00 P. M. 

The morning session on Friday was much the same as at Minnesota, 
but some new handouts were included which seemed to make it go better 
A couple of new ideas were also added to the PERT/Cost presentation 
v/hich helped to strengthen it. The afternoon session went according 
to schedule and was ended at 3:00 P. M. 

University of Pittsburg h. (Monday-Tuesdey , April 17.-13) • The 
lectures were held in the Supper room of Shcnley Park (the Student 
Union building). The set-up consisted of chairs with no tables, 
which presented some difficulty in doing the practical exercises. 




It appeared that much effort had gone Into the publicity since there 
were about i, 5 persons attending the initial session. The participant 
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came from many places including the State Department of Public 
Instruction as well as from some of the small universities and private 
consulting firms and/or similar agencies. About 30 participants 
stayed through the fourth session. 

Due to some difficulty in getting audio-visual arrangements set 
up, the first session started about 15 minutes late. Further, be- 
cause of difficulties with Dr. Cook’s voice, Mr. Stahl made the morn- 
ing and afternoon presentation on Monday. 

The new material on scheduling did not arrive until late Monday 
morning so parts of it were not available for the morning session 
but were available for the afternoon. In working out the revised 
Practical Exercise No. 5* an error was discovered which caused some 
minor confusion among the pa rt i c i pants . A connection was immediately 
noted, but still perhaps left some persons slightly confused. The 
staff felt that Practical Exercise No. 6 may turn out to be somewhat 
confusing since some activities were changed but other changes which 
would logically follow were not made. The afternoon session went 
according to schedule. 

The third session was conducted according to the revised script 
as presented at Iowa. This revised script appears to work satisfac- 
torily, but there seems to be a demand on the part of the participants 
for more detail on the mechanics of working out the problems. Generally, 
the session was on schedule except for the coffee break which took a 
bit longer, hence there was not time for questions in the morning. The 
afternoon session, however, went as in the past and finished about 
3:00 P. M. 




Several events at this lecture served to point 



out problems which 
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could have been avoided by proper preparation. Again, as was the 
case at previous lecture sites, the first session started late be- 
cause the lecture staff failed to check the audio-visual equipment 
in advance. Or. Cook*s loss of voice and consequent inability to 
deliver the lectures on the first day, pointed out a problem which 
could have been quite serious. Fortunately, Mr. Stahl knew the mat- 
erial and was able to step in and handle the lectures by himself. 

Had Mr. Stahl not been prepared for such an emergency, the lectures 
would have had to have been canceled, causing much inconvenience and 
loss of money to both the project and the participants. 

Michigan State University . (Monday-Tuesday , April 19-20). The 
lectures were held in the Kellogg Center which was filled to its 50 
person capacity for the first session. The registration procedures, 
which included advance registration, were well handled and the facil- 
ities and physical arrangements were more than adequate. Approximate- 
ly a dozen of the first session participants were individuals attend- 
ing a Vocational Education Research Conference which started that 
evening and continued the rest of the week. Many of these persons had 
heard about the lectures from other sources and decided to combine 
the two events. Persons also attended from the State Department of 
Public Education as well as from other universities in the area. 

The morning session on Monday was conducted according to the 
revised script and seemed to go quite well. The afternoon session 
also went according to the revised script and was completed about 
4;00 P. M. 




The number of persons attending the final two sessions dropped 
to about thirty due primarily to the absence of the persons attending 
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the Vocational Education Research Conference. The session started 
on time in the morning and the afternoon session ended at the 
usual 3:00 P. M. The first two sessions were also conducted according 
to the regular revised script. 

Indiana Universi ty . (Monday-Tuesday, April 26-27). The lectures 
at Indiana were held in the new University School which is located 
immediately adjacent to the main campus. Without condemning any one 
person in particular, the arrangements at Indiana University were 
inadequate. The area coordinator submitted a list of letters sent out 
in the area and it indicated a rather limited potential audience. In 
addition, some participants revealed that information about lectures 
was not received until the end of the week just prior to the lecture. 

The room arrangements were satisfactory except that the facility 
provided was a large lecture room which was not needed with the small 
size of the group who attended. Also the room was cold, causing many 
participants to feel uncomfortable. Some participants reported that 
in checking with several offices on the campus, no one was able to 
tell them quickly where the conference was being held and thus were 
late arriving for the first session. 

The morning session started about a half-hour late because the 
project staff had to secure audio-visual equipment. Attendance was 
small, with only 13 registrants. During the afternoon session only 
eight persons were in attendance, possibly due to the fact that the 
regional coordinator left during the lunch break for an out-of-town 
conference. 
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The morning session started on time Tuesday with only one change 
in the regular presentation. More time and attention was given to 
other applications of PERT in the field of education than those stem- 
ming directly from the PERT Project. This change was instituted 
on a trial basis to see how the audience would react. The change 
was made because many participants at other lecture sites had indi- 
cated that they would like to know more about what was going on in 
other parts of the country. Reaction to the change appeared to be 
pos it i ve. 

No major changes were made in the presentation of the final 
session and the final lecture in the series was concluded about 3:00 
P. M. 

Summary . The technique of recording reactions immediately turned 
out to be a fairly good device for receiving evidence and is recommended 
for similar kinds of efforts. While its use might be limited by a 
type of "first impressions" atmosphere, the lessons learned and 
recorded at the time were quite helpful in revising the lecture ser- 
ies as they progressed. 

Cost Analys is 

The rather unique method of dissemination incorporated into the 
project required a rather substantial amount of funds to carry out. 
Because of this situation, a decision was made to attempt to include 
some form of cost/effectiveness analysis as part of the evaluation of 
the lecture series. The purpose of this section is to present the 
procedures employed and results obtained with regard to a cost analy- 
sis of the dissemination lecture series. 




The contract negotiated with the Office of Education allocated 
a total of $42,622 for the overall project. It would have been 
highly desirable to secure a rather firm estimated cost figure for 
the dissemination lectures. The selection of host institutions 
required both time and resources and did not commence until after 
the contract was negotiated. Since the location of the lecture sites 
would be crucial to any estimation of expenses such as travel and per 
diem, it was not possible for the project staff to submit an itemized 
estimate of dissemination lecture expenses. Once the lecture sites 
had been selected and related arrangements established, it was possible 
to secure an estimated cost for the series of lectures. The estimated 
cost figures are shown in Table 4. 

The manner in which the travel and other arrangements were made 
for the dissemination lecture made it difficult to compute the actual 
costs associated with conducting each regional lecture. The schedule 
was set up so that a minimum amount of travel would be involved be- 
tween any two lecture sites. Thus transportation costs to the Los 
Angeles sites were relatively low since the lecture there was con- 
ducted one day after the lecture at Berkeley. For purposes of actual 
costing, such trips were treated as a single five-day session. The 
actual costs derived on this basis are presented in Table 5. 

In deriving the actual costs presented in Table 5, the cost of 
materials (e. g. , movies, transparencies, handouts, worksheets, and 
related items) have been included as an average for all lectures rather 
than differing from site to site as they actually did since it was 
almost impossible to break out such costs on a per site basis. The 
totals in the staff wages column of Table 5 were determined by multi- 
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Table 4 - Estimated Costs for Dissemination Lecture Series 
(Excluding salary and material costs) 



Location Number of Number of Travel Per Diem Total 
days staff 



University of 
Cal i fornia 



and 

Un ? vers i ty of 
Southern Cali- 
fornia 


7 


3 


$798 


$336 


$1,134 


Florida State 
Uni vers ity 


4 


3 


345 


192 


537 


Syracuse Univer- 
s i ty 
and 

Columbia Univer- 
s i ty 


6 


3 


234 


288 


522 


Wash i ngton Un i ver- 
s i ty 
and 

Un? vers i ty of 
Iowa 


6 


3 


315 


288 


603 


University of 
Minnesota 
and 

Uni vers i ty of 
Wi scons i n 


C 


3 


294 


288 


528 


University of 
P i t tsburgh 


3 


3 


84 


144 


228 


Michigan State 
Uni vers i ty 


3 


3 


141 


144 


285 


1 ndi ana Un i vers i ty 


3 


3 


144 


144 


288 



Totals 35 - $2,355 $1,824 $4,179 
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plying a dally rate for each of the lecture staff members conducting 
the lecture by the number of days spent traveling to the site, 
conducting the lecture, and either returning to the project head- 
quarters or traveling to the next lecture site. The wage figure 
was based on the dai T salary of the lecture staff using a five-day 
week. Per diem costs included such items as meals, hotels, telephone 
calls fo*- calling project headquarters during the lecture series, and 
for baggage charges at the hotel where the lecture staff stayed. 

Travel costs included air fare, taxi or limousine travel to and 
from airports, and car rentals. 

Direct comparison between the estimated figures shown in Table 
4 and the actual figures shown in Table 5 are not entirely possible; 
since different procedures were used in obtaining the data. For 
example, the estimated costs were made on the basis of three persons 
traveling to ench lectuie site. This actually happened on only two 
occasions as noted in Table 5. 

Because of the difficulty of making the direct comparison between 
estimated and actual costs, some other means of arriving at a useful 
cost/effectiveness analysis was deemed desirable. The procedure 
employed was to relate the cost of the lecture series to the number 
of participants served or to establish per participant cost . Using 
the actual cost figures presented in Table 5 and dividing by the num- 
ber of participants attending the lectures at each site, per partici- 
pant costs were established for each combination of trips plus for the 
total series of lectures. These costs per participant are shown in 




Table 5. 
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NOTES: 1. All costs rounded to nearest dollar amount 

2. Per diem cost includes hotel, meals, and miscellaneous 
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While costs varied from session to session, the average costs 
were approximately $26 per participant. Evaluation of this figure 
is difficult to make s f nee there are no previous standards against 
which to make comparison. Each reader can make his own interpreta- 
tion as to whether or not the direct cost per participant is too 
high or low for substantive information presented by the dissemina- 
tion lecture series employed by the project. 

S umma ry 

The purpose of this chapter has been to present evidence of both 
a quantitative and qualitative nature relative to the general effec- 
tiveness of the lecture presentations as well as the overall dissem- 
ination lecture series. Evaluations were secured from participants 
attending the sessions, from the director at the time each session 
was r^rried out, and from the records • the project with regard to 
a cost analysis of the lecture series. 

Based upon the evidence presented from the above three sources, 
the project staff believes that the dissemination lectures accomplished 
their prime objectives of reaching a segment of the educational 
research community and introducing that community to the concept of PERT. 
Participant evaluations indicate^ that the conjept was adequately pre- 
sented but needed more direct relationship to educational practice. 

The director's reactions focused primarily upon an administrative eval- 
uation of the presentation. Evidence here indicated that physical 
arrangements, techniques of presentation, and functions of liaison per- 
sonnel were important in conducting such dissemination efforts. Based 
upon original plans for the project that called for a possible 
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1,200 participants at the lecture series, the actual cost per par- 
ticipant of $26.00 each seems extremely high. In actuality, the 
figure might not be unreasonable since there is no real way of estab- 
lishing the validity of the anticipated participants at each of the 
sess ions. 

The data presented in this chapter represents immediate reactions 
The effectiveness of the lecture series might be long range. For 
this reason, a follow-up study was conducted approximately six months 
after the completion of the lecture series. The results of this 
study are reported in Volume II of this two volume series. 
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CHAPTER V 



SUMMARY AND RECOMMENDATIONS 



The purpose of this chapter is to summarize the purposes, pro- 
cedures and evaluation results of the dissemination lectures. A 
set of recommendat ions relative to methodological problems encoun- 
tered while conducting the lectures based upon observations is also 
included in this chapter. Conclusions on the long-range effective- 
ness of the lectures were derived mainly from the quest i onna i re which 
was sent out after completion of the lectures and therefore are 
included in Volume II of this report. 

Purpose 

Broadly speaking, the purpose of the dissemination lectures 
was to inform the educational research community about PERT, a 
management technique developed for planning and controlling research 
and development activities, and to encourage its use. 

More specifically, the lectures were designed to accomplish two 
main objectives: (l) to reach the largest possible number of actual 

and potential educational researchers, and (2) to disseminate infor- 
mation to them on the principles and utilization of the PERT tech- 
nique. 
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Procedures 

Several different activities were initiated by the PERT staff 
in an attempt to realize the two objectives mentioned above. The 
first objective of reaching a designated audience demanded both the 
selection of suitable locations for conducting the lectures and 
appropriate measures to secure the type of audience which was 
desired. Consequent 1 y, twelve universities which fulfilled certain 
criteria concerning the size and composition of a potential audience 
within easy traveling distance were selected as lecture sites. 

A liaison person was then selected at each lecture site to assist 
i r> coordinating efforts between the project staff and the host insti- 
tution. These liaison men were responsible for the physical arrange- 
ment at their lecture site and for efforts to locally publicize the 
lecture in their area. 

In addition to local efforts by the liaison man, publicity was 
also secured on a national level by the project staff. Announcements 
and descriptions of the lectures were placed in numerous professional 
journals and were also made available to attendees at the 1 965 AERA 
convention in Chicago. 

The second objective of effectively disseminating information 
was related to the first objective in that the coordinating efforts 
(publicity, securing visual aids equipment, reserving conference 
space) of the liaison personnel made it possible for the project 
staff to effectively communicate with the participants at the various 
lecture sites. The effectiveness of the lectures, however primarily 
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depended upon the organization and presentation of the material by 
the project staff. The content material was divided into four parts 
or sessions. These sessions were arranged in ordered sequence from 
a general overview of PERT to a presentation of PERT/COST, a special- 
ized analytic technique. Participant responses to a questionnaire 
administered at the completion of each regional lecture were the 
primary sources of feedback on the effectiveness of the presentation 
for the project staff. The staff made several changes between 
lecture sites in both content and method of presentation. Changes 
which were well received by the participants were retained as part 
of the regular presentation. Thus the materials and presentation 
form used at a particular lecture site depended somewhat upon how 
effective they were at the previous lecture sites. 

The participant evaluations from each lecture site, a cost 
analysis index based on per capita costs for each regional lecture, 
and a short evaluation which the project staff completed after each 
lecture were all used to evaluate the extent to which the two main 
objectives of the dissemination lectures were reached. The evalua- 
tion section of this report reveals that in general both objectives 
of the lectures were met, although publicity arrangements in some 
instances were unsatisfactory and the first objective was not met 
at several lecture sites. 
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Recommenda t i ons 

Based upon the description of the various phases of the 
dissemination lecture project as presented in the previous chapters 
and based upon personal experience gained by the project staff in 
the actual conducting of the lectures, the following recommendations 
are offered for conducting future dissemination projects of this 
type. 

1. The selection of the area coordinators should be made 
carefully since this person appears to be quite influential 
in determining the size and composition of the audience. 

Of particular importance is the correla t ion whi ch seemed 
to exist between interests of the area coordinator and the 
interests of the persons who attended the lecture in that 
area . 

2. The confusion over responsibility for regional publicity 

at several lecture sites indicates that a form letter should 
be sent to the area coordinators asking them what publicity 
arrangement they have made and emphasizing their responsibi- 
lity for securing an audience which generally fits the 
criteria specified by the project staff. 

3. Projection equipment should be checked in advance of each 
regional lecture to assure smooth operation. Visuals 




should be checked to assure readability, espec i a 1 1 y when the 
size of the facility for the lecture is not known in advance. 



59 



4. More workshop- 1 i ke activities should be included so that 
participants can be actively involved as soon as possible 
in the topic being presented. 

5. Practical activities should be introduced early into the 
lecture sequence so that participants do not have to sit 
through "talked-at 11 sessions. 

6. Some type of lesson outline set up so that notes taken 
would be very useful. High points could be noted and 
space provided for the participants to fill in the deta i 1 . 

7. I f at a 1 1 poss ible a dry-run should be made wi th an audi ence 
very similar to the projected audience in order to try out 
the length of presentations, eliminate repetition of ideas 
and concepts, i nterweave aud i o-v i sua 1 materials, and screen 
other planned activities. 
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1. Dissemination Center Evaluation Worksheet 

2. List of Publications Carrying Publicity Announcemen 

3. Publicity Information Sheet 

A-. Participant Evaluation Questionnaire 
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PUBLICATIONS CARRYING ANNOUNCEMENTS OF LECTURES 

1 . Phi Delta Kappa n 

2 . American Psychologist 

3. Newsl etter , American Educational Research Association 

4* Newsletter , National Council on Measurements in Education 

5. Newsletter , Division 15^ American Psychological Association 

6. Theor/ Into Practice, The Ohio State University 

7. Newsl etter , Association for Educational Data Systems 

8. Newsl etter . Laboratory for Experimental Design, College of 

Education, University of Wisconsin 

9. Coming Events Column, Data Processing Magazine 

1 0. Journal of Counsel inq Psychology 

1 1 . Personnel Guidance Journal 

12. Communications of the Association for Computing Machinery. 
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The Bureau of Educational Research and Service of The Ohio 
State University announces a series of twelve lectures to be held 
during 1965 the purpose of which is to disseminate information 
about the applicability of PERT to educational research and develop- 
ment activities. PERT, an acronym for Program Evaluation and Review 
Technique, [ s a recently developed management technique considered 
to be a highly useful tool for the planning and conducting of 
research and development activities characterized by a "once-through" 
nature. The dissemination lectures are being supported by a grant 



from The Cooperati 


;ve Research Program of the U. S. Office of Educa 


t i on. 








The specific 


dates 


and locations for the lectures are as 


fol 1 ows: 








March 


1 - 2 , 


1965 


University of California (Berkeley) 


March 


5 , 


1965 


University of Southern California 








(Los Angeles) 


March 


15 - 16 , 


1965 


Florida State University 


March 


22 - 23 , 


1965 


Syracuse University 


March 


25 - 26 , 


1965 


Columb ia University 


March 


29 - 30 , 


1965 


Washington University (St. Louis) 


April 


1 - 2 , 


1965 


State University of Iowa 


April 


5- 6, 


1965 


University of Minnesota 


April 


8- 9, 


1965 


University of Wisconsin 


April 


12-13, 


1965 


University of Pittsburgh 


April 


19-20, 


1965 


Michigan State University 


Apr i 1 


26-27, 


1965 


Indiana University 



In addition to the lecture series, a workshop emphasizing appli- 
cation to on-going and proposed projects will be held at The Ohio 
State Un i vers i ty Campus on Hay 10, 11, and 12, 1965. 
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Further information regarding the PERT project, the dissemina- 
tion lectures, and the workshop may be obtained from Or. Desmond L. 
Cook, Director, The PERT Project, 4l West 11th Avenue, Columbus 10, 
Ohio. 
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The PERT Project 

Bureau of Educational Research and Service 
The Ohio State University 
41 West 11th Avenue 
Columbus, Ohio 43210 

EVALUATION QUESTIONNAIRE 

In the interest of improving the lectures on “The Program 
Evaluation and Review Technique, 11 we would appreciate your opionions 
and judgment as expressed by the following questions. You are asked 
to sign this questionnaire in the place provided at the end. 

1. To what extent do you feel that the Not at all 

material covered in the lectures is Very little 

pertinent to your work in the area Somewhat 

of educational research and develop- Significantly 

ment? Please give reasons for your A great deal 

answer. 

2. To what extent was the lecture material Directly 

related to your duties? Indirectly 

Not at all 

Not at all 
Very 1 i tt 1 e 
Somewha t 
Significantly 
A great deal 

None 

Very 1 i tt 1 e 
Somewhat 
Si gni f icant 
Great 

None 

Very 1 i tt le 
Somewha t 
Si gni f icant 
Great 

Too easy 
Easy 
Avera ge 
D i ff icul t 
Too di ff icul t 



O 




6. What was your reaction to the com- 
plexity of the practical problems? 
Comments . 



How effective was the use of visual 
aids (slides, movies, handouts) 
in the lectures? Comments. 



Of what value were the practical 
exercises to vou? Comments. 



3. Do you anticipate using the material 
presented in the lecture? Please 
expla in. 
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7. How do you feel about the level of Too detailed 

detail in the lectures? Comments. Detailed 

Average 

Gross 

Too gross 



8. What would be your overall rating Excellent 

of the lectures and how do you Good 

think they can be improved? Please Average 

be specific as possible. Fair 

Poor 

9. If the lectures were to be shortened First 

by one session, which session would Second 

suggest omitting? Third 

___ Fourth 

10. Would you be interested in lengthen- Yes 

ing the lecture so PERT/COST could be No 

considered in more detail? 

11. Which session did you consider to First 

be the most informative? Second 

Th i rd 

Fourth 

12. How would you evaluate the instructors Excellent 

with respect to their presentation, Good 

knowledge of the subject and their Average 

ability to explain and clarify points? Fair 

Poor 



NAME 

POSITION 

ADDRESS 

LECTURE LOCATION 
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Table 6 - Participant Evaluation of Lecture Relevancy to Work 
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Table 7 - Participant Evaluation of Lecture Relevancy to Duties 
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Table 8 - Participant Evaluation of Anticipated Use of PERT 
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Table 9 - Participant Evaluation of Teaching Aids 
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Table 10 - Participant Evaluation of Practical Exercises Value 
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Table 11 - Participant Evaluation of Practical Exercise Complexity 
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Table 12 - Participant Evaluation of Lecture Detail Level 
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Table 13 - Participant Overall Rating of Lecture 
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Table 14 - Participant Recommendation on Session Elimination 
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Table 15 - Participant Recommendation Regarding PERT/COST 
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Table 16 - Participant Evaluation of Most Informative Session 
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Table 17 - Participant Evaluation of Instructors 
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